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Cynthia G. Parshall 
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Judy Ritter 

Institute for Instructional Research and Practice 
University of South Florida 



Abstract 

One of the many growing areas of computer applications is that of computerized assessment. While 
computer-based tests might be as simple as computerized versions of paper-and-pencil exams, more 
innovative applications also exist. Innovative, computer-based items provide features not typically 
available in paper-and-pencil tests. Examples of assessment innovations include: the use of 
graphics or sound, some measure of interactivity, a change in the means by which examinees 
respond to items, and the application of computer-scored partial-credit models. This session will 
include a demonstration of innovative computer-based items, together with a discussion of the 
developmental process. Issues in evaluating the quality of new forms of assessment will also be 
addressed. 
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Innovations: Graphics, Sound, and Alternative Response Modes 

There are many forms which testing or assessment might take. These methods of 
assessment can be conceptualized along a continuum, with multiple-choice items at one end and 
performance tasks at the other. Various item types or test formats may fall between these two 
extremes. In general terms, those forms of assessment closer to the “multiple-choice” end of the 
scale will have potential advantages such as high reliability, broad content coverage (because 
more multiple-choice items than performance tasks can be administered in a given time period), 
and low-cost. The accompanying disadvantages will be the risk of a decontextualized 
assessment, and an emphasis on information that is easy to assess, rather than important. 

Assessment methods approaching the “performance task” end of the scale will have 
opposite advantages and disadvantages. These assessment tasks have the important advantage of 
a higher potential for relating closely to the real-world skills of interest. However, they may also 
have problems with low reliability, excessive dependence on a narrow range of content, and may 
be relatively expensive to develop, administer, and score. 

An ideal form of assessment would include as many of the relevant measurement 
advantages as possible, while retaining few of the disadvantages. Within the last few years, a 
great many efforts have been underway to develop new and relevant forms of assessment. 

Quality concerns for these developments include an emphasis on measuring important skills, 
developing appropriate scoring models, and augmenting the diagnostic information obtained. 

One area in which promising work is occurring is that of computer-based innovative items. 
Using the idea of the continuum presented above, it is possible to consider computerized 
assessments at both extremes. There are computer-based tests that consist entirely of discrete, 
multiple-choice items, and others that provide computerized performance assessments. 

The majority of computerized assessments in current use consists of traditional, multiple- 
choice, text-based items. As is often found when a new technology first becomes available, the 
initial applications look very similar to the processes they are superseding. In many cases, 
computer-based tests are developed entirely from existing standardized paper-and-pencil exams. 
(This description of computerized testing does not address the issue of computer adaptive tests. 
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Theoretically, adaptive administration of items could occur using either multiple-choice or 
performance items.) 

Towards the other end of the scale, computerized performance tasks include applications 
such as direct measures of office skills (e.g., word-processing and key-punching). At the outer edge 
of performance assessment on computer, come computerized simulations. Examples of this include 
flight simulators for assessing pilots, on-screen engine mock-ups for testing the operators of large 
machines, patient-management software for testing medical personnel, and architectural design for 
assessing architects. The “simulation-based” exams tend to be developed in specialized fields, take 
a large developmental effort to produce, and require extensive scoring rules. The number of these 
applications is growing, but relatively few of them are currently operational in comparison to the 
more traditional item types. 

Between these two scale end-points fall what may be termed innovative items, the topic of 
this paper. This type of item provides some feature not typically available in paper-and-pencil tests 
and is also delivered through and scored on computer. These innovative items do not include full- 
blown simulations and will typically involve less development time and more limited scoring 
rubrics. Ideally, these measures will retain the quality measurement attributes of reliable, 
inexpensive scoring while incorporating features that increase the validity and utility of the 
assessments. 

Computer-administered items may be termed innovative for a number of reasons. 
Innovations might include the use of graphics or sound, some measure of interactivity, a change in 
the means by which examinees respond to items, or the application of computer-scored partial- 
credit models. These items may be innovative through features included in the item stem or 
through alternative response modes required of the examinee. In every case, the rationale for these 
innovations is improved measurement. 

Some innovations provide examinees with a better context for assessment. An example of 
this added “context” might be a test that provides on-line information that a student can refer to and 
utilize when responding to an item. Another example is an item that prompts an examinee to 
indicate a sequence for a set of events. When this type of item is computerized, the computer can 
display a reordering of options, based on the examinee’s response. 
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Other innovations reduce the likelihood that an item can be answered correctly through 
guessing. For example, a math item might require the examinee to type in a response, rather than 
select from 4-5 given options, or a reading item might prompt the examinee to select a word or 
phrase from all possible phrases within a reading passage, rather than from a limited set of options. 

Still other innovations increase the information obtained per item. This might occur by 
having the computer keep track of the process an examinee goes through to produce a product. For 
example, the set of steps taken by an examinee in a science activity could be scored, using a partial- 
credit scoring model. In these and other innovative applications, the additional features should 
improve measurement in a substantive way, rather than simply adding glitz to a traditional test. 

Purpose 

The purpose of this study was to investigate several computer-based innovative item types. 
A set of items was developed in conjunction with a college-level introductory computer course and 
was fieldtested on 93 of the course’s students. The item types were evaluated on three dimensions: 
(1) the technological functioning of alternative features, (2) reactions from examinees to the 
innovations, and (3) the psychometric performance of the items in comparison to existing paper- 
and-pencil assessments. 



Descriptions of Innovations 

The computer-based test consisted of four sections, each containing four items. A single 
item was displayed on a screen at a time. Within a section, an examinee was free to page back and 
forth between that set of items, but the examinee could not return to a preceding section. 

The features under investigation included innovations both in item stems and in response 
modes. Innovations in item stems involved the use of sound or graphics not easily provided in 
standardized paper-and-pencil testing. Alternative response modes included requiring examinees to 
select multiple responses, to click on a screen graphic, and to play one or more sounds before 
selecting an answer. All of the items were scored dichotomously (e.g., either correct or incorrect), 
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although possible adaptations using partial credit may be considered for future versions and 
additional items. 

Although this exam was designed for a computer class, and thus for students who had been 
exposed to a variety of computer applications, it was still considered desirable to limit the amount 
of “learning” necessary to take the exam. A decision was therefore made that each section of the 
exam would cover a specific area of course content and would also be limited to a single mode of 
responding. The first two sections consisted primarily of items adapted from the current paper-and- 
pencil examination and items in the remaining sections were primarily created for computer 
administration. 

Section 1 addressed educational applications of computers. This section consisted of text 
only; however, it was designed so that examinees could select more than one of the response 
options available (i.e., “Select all that apply”). (See Figure 1 for a sample item). Section 2 
measured the topic of CD-ROMs and multimedia applications. This section included graphics that 
related to text in the item stems. Examinee responses were made by clicking on a box next to one 
of the response options listed (Figure 2). Section 3 covered the topic of databases, used on-screen 
graphics, and required the examinee to select a response by clicking on part of that graphic (Figure 
3). Finally, Section 4 addressed the use of sound in computer applications. Each item in this 
section contained buttons that the examinee clicked to hear sounds. After listening to the sounds 
(music, beeps, or voice, depending on the item), the examinee indicated a response to the item by 
clicking on a box (Figure 4). At the conclusion of the items, an on-screen survey was presented to 
examinees. The survey addressed examinees’ reactions to the computer administration of the 
exam, as well as to the innovative features of the items (Figure 5). 

Methods 

The computerized innovative items were fieldtested on a number of volunteers. In late 
November, 1995 the computer exam was taken by 93 out of approximately 300 students who were 
completing a college-level introductory computer course. All examinees were volunteers, who 
received course credit for their participation; however, their performance on the test did not affect 
their course grade. 
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The computer exam was designed to address content covered in the second half of the 
course, and was partially comprised of items adapted from the course’s second examination. The 
computer administration took place approximately two weeks before the related paper-and-pencil 
exam was given. No counter-balancing could be carried out due to the fact that in this course, 
immediately after an exam is administered and the answer sheets are collected, an oral review of all 
items is conducted. Since approximately half of the items on the computer-based exam were close 
adaptations of items on the paper-and-pencil exam, it seemed to be too problematic to attempt 
administering the computer exam after this review. One way around this problem was a planned 
comparison of the performance of those students who took both the computer and paper-and-pencil 
exams, to the performance of the remaining course students who took only the paper-and-pencil 
exam. (For purposes of analysis, the paper-and-pencil only group consists of those students who 
completed the course, who did not participate in the computer-exam study, and who earned a final 
course grade of D or better.) 



Results 

Technological Issues 

There were three primary areas of interest in the evaluation of this assessment. The first 
related to technological issues. The goal here was to isolate (and correct) any problems which 
might arise due to the use of sound and graphics. Possible problems included poor quality display 
of graphics or reproduction of sound on test administration computers. It was also a concern that 
the addition of these elements could make the computer file size prohibitively large or cause the 
program to run at an unacceptably slow speed. These technological questions were considered as 
the computerized exam was developed, and during the field test. 

The computer-based items were developed using the program Authorware. The final 
computer test file consisted of 59 screens and was 1.8 megabytes in size (i.e., larger than would fit 
on a diskette without compressing). Of these 59 screens, 10 were devoted to instruction and 
practice items and 18 pertained to the concluding survey. The test included 18 sound files (.WAV), 
that ranged in size from 5 kilobytes (a half-second burst of laughter) to 134 kilobytes (a 6-second 
musical piece). Many of the graphical images were created within Authorware (e.g., green 
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background, directional arrows, etc.); others were imported as bitmap files (.BMP) and pasted onto 
the appropriate screens. Improvements in efficiency are probably possible, as no particular effort 
was made to be memory efficient in the development of the computerized exam. However, even in 
its current form, the computer file ran without dragging or delaying. 

One unsurprising conclusion of this study is that a developer must standardize on some 
minimal level of computer hardware and software. For some computerized tests, this could mean a 
286 processor with a black-and-white, text-based monitor, and no sound capacity. A Windows- 
based, 386 computer with an SVGA monitor and sound card was the standard computer used in this 
study. Based on this platform, no problem was found with any graphical display. Unexpectedly, a 
few of the sound files had too much static when played on some test administration computers. 

This may have resulted from sound files with lower sampling rates, and/or computers with poorer 
quality sound cards. This problem would need to be corrected before operational testing occurred. 
Examinee Reactions 

The second evaluation issue pertained to examinee reactions to this form of assessment. At 
the conclusion of the items, an on-screen survey was presented to examinees. This survey 
addressed examinees’ reactions to the computer administration of the exam, as well as to the 
innovative features of the items. Appendix A contains a list of the survey questions, along with the 
number of examinees who made each selection. 

As responses to the survey show, almost equal numbers of examinees rated themselves as 
having entered the course with “very little” (n=48) or “some” (n=44) computer experience. For this 
computer oriented class, a majority of the students (n=60) indicated a preference for a computerized 
test rather than a paper-and-pencil version. For other courses, that number dropped (n=37), and the 
number of students preferring paper-and-pencil testing increased (n=44). 

The survey was structured so that examinees who indicated they had a problem (or 
suggestion) were routed to another screen where they could type in additional information. A 
majority of the examinees (n=50) responded “no” to every item that asked if they had problems in 
taking the exam. No problems were noted by any examinees in the use of the mouse or in the use 
of the Previous and Next navigation buttons. 
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Some problems were identified for each of the four exam sections. The intention of the 
survey questions was to isolate difficulties examinees may have had understanding test directions 
or using the computer test software. Based on a misunderstanding by some examinees as to the 
purpose of the survey, many of the responses were actually related to the course, or to the 
examinees’ confidence in their own knowledge level, rather than to the computer innovations. For 
example, 25 examinees indicated they had a problem when taking Section 1 . However, many of 
these problems turned out to be a variation on “I did not remember what everything meant.” , “not 
remembering some of the information from class”, or “wasn’t sure of a few answers”. Other 
examinee comments are too ambiguous to be helpful. Examples of ambiguous responses included 
“unknown”, “not familiar”, “just a little uncertainty”, and “I did not understand what some of the 
questions were asking”. (Did the examinee who provided this last statement have difficulty with 
the items or with the directions?) 

Two pertinent responses to Section 1 addressed the format of items in that section, in which 
examinees were prompted to “Select all that apply” (e.g., “I had a problem answering the questions 
knowing that there was more than one answer. I always get confused in these situations.”). 

For Section 2 (which incorporated graphics into the item stems), far fewer problems were 
noted. Of the six examinees who indicated a problem with this section, most commented about 
problems with their own knowledge (e.g., “I wasn’t sure of the capacity of a cd rom”). One 
examinee instead appeared to indicate difficulty with the directions (“I was not sure what I needed 
to do.”). 

Of the nine problems noted for Section 3, only one was clearly related to taking the 
examination. The examinee indicated “after going to the next question I was fine”. Items in this 
section required examinees to respond by clicking directly on a graphic, and it appears that this 
examinee required a second example of this item format before that method of responding was 
clear. 

Only five problems were noted for Section 4, most of which were directly related to the 
incorporation of sound, which distinguished these items. Two examinees expressed concerns about 
being able to hear the items. (One wrote, “It was hard to hear some of the sounds”. The second 
indicated, “I am hard of hearing and I find it a little hard to hear some of the sounds. I feel that in 
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an actual test it would greatly influence my ability to successfully complete the test.”.) Another 
examinee indicated difficulty with this section, “Could not get sound to work for the first three 
questions”. (This problem was noted during the field test for a few examinees, all of whom had 
neglected to take the practice items in the instruction screens. An obvious correction to this 
problem is to structure the practice screens so that examinees are required to respond appropriately 
before proceeding.) 

Finally, general comments were provided by 23 examinees. Again, some of these applied 
to the course rather than to the exam. Those that applied to the computer exam tended to be 
positive, (hey also tended to be at least as much a reflection of the examinees’ reaction to the 
computer mode as to the innovative features used in this instance. Several comments were similar 
to the remarks “I felt that a computer test might be intimidating, yet it was not; it’s actually fun!” 
and “I enjoyed taking this test”. In a more discriminating vein, one examinee stated, “I think the 
use of the computer in testing might help clarify the question (sic) in this domain. Sometimes 
visualizing can be difficult. It should make testing some concepts more practical.” While another 
examinee suggested a concern, “Taking tests with paper/pencil allows me to change my answers as 
often as I need to. I have had a course where the tests were administered by computer. It was okay, 
but making a mistake was costly.” Finally, the issue of sound was re-addressed with praise “The 
audio portion of the exam was great.” and concern “I feel that a computerized testing procedure is a 
great idea except for the sound portion which I feel may hinder some of us with hearing 
disabilities”. 

Psychometric Functioning 

The final evaluation considered in this study concerned the psychometric functioning of 
these items. Because the participants in this study were volunteers, it seemed reasonable to 
compare their performance in the course to that of the nonparticipants. In Table 1 presents the 
means for these two groups on the paper-and-pencil test (Exam 2) and on the course as a whole 
(Course Total). A visual inspection of the table indicates that the volunteers are roughly 
comparable to the nonvolunteering students in terms of course material mastery. 

Table 2 provides all the item p-values for the computer-based innovative items. Where 
those items were adapted from paper-and-pencil items, p-values on the paper-and-pencil versions 
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are also provided, both for those students who had taken the computer test, and for the students who 
took only the paper-and-pencil exam. Examinees’ responses to those items that were presented in 
both modes can be compared directly. 

Inspection of the item p-values reveals a difficulty, apparently related to a time effect. The 
computer exam was administered first, and was administered approximately two weeks prior to the 
paper-and-pencil exam. Although the students who volunteered to take the computer-based exam 
appeared to make every effort to do their best, they probably had not studied prior to the exam. By 
the time the paper-and-pencil exam was administered (effecting the course grade for all of the 
students), it appears that much further studying and preparing had occurred. Thus, the largest effect 
apparent in Table 2 is the improved performance on the paper-and-pencil version of the exam. 
Consideration of this effect in light of examinees’ comments such as “I couldn’t remember...” 
suggest that this effect is more likely related to preparation than to mode of administration. 

For those students who took both the computer and the paper-and-pencil exams, further 
analyses of the items were conducted. The objective of the item analyses was to investigate item 
performance, as compared to an existing criteria. However, it was not necessarily desirable that the 
computerized assessment produce results identical to other evaluations of student learning (or 
extremely high correlations). Rather, appropriate functioning of the items and item types would be 
supported by a general positive relationship between the different modes of assessment. 

First, correlations between the items that appeared in both modes were computed across 
mode. The resulting phi coefficients were all positive. About half of the coefficients were quite 
modest (r < .10), with the remainder ranging from r = .14 to r = .34. Next, point biserials of all of 
the computer items to the examinees’ numerical score on the course as a whole (Course Total) were 
calculated. These results also showed a positive trend (a single negative correlation of r = -.07 was 
found). Six of the point biserials fell between r = . 1 1 and r = .20, and the 3 strongest correlations 
were .24, .27, and .41. The strongest relationships were found between items in Sections 2 and 3 
and the Course Total, a not surprising result, given the content of those sections (Graphics and 
Databases) and the relative weights these topics assumed in the course total. 

Most of the items adapted from the paper-and-pencil exam are presented in Sections 1 and 2 
of the computer exam. The items in Section 1, although adapted from existing items, contained 
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some important differences. The computer format for these items is “cleaner”, with a much simpler 
set of instructions. The paper-and-pencil format for these items provides a certain amount of 
cueing, not present in the computer format. The lack of counter-balancing in this study makes more 
detailed analysis impossible, but these format changes may yield improved measurement, and seem 
worthy of further investigation. 

The mean score for the 4 items in Section 1 was 1.27, making it the most difficult of the 
computer test sections. (This increased difficulty may have been related to the content, to the item 
type, or simply to the fact that it was the first section.) For Section 2, the mean score was 2.25. As 
compared to Section 1, the items were more similar to their paper-and-pencil counterparts, 
including a graphical representation of each item’s topic, but (unlike Section 1) requiring 
examinees to select only a single “best” answer. 

For the most part, the items in Sections 3 and 4 of the computerized test were not present in 
the paper-and-pencil exam (and, in the case of Section 4, could not be). The item p-values for 
Section 3 suggest that these items were fairly easy (with a section mean of 3.06). Aside from the 
novelty of the item response format (clicking on a graphic of a database), the items might be 
expected to be easy since they assessed the most fundamental database topics covered by students 
in this course (e.g., search, sort). Since examinees demonstrated an ease and comfort with this item 
type, items designed to tap more difficult content could easily be developed. 

In Section 4, most of the items have very high p-values. The mean for this section was the 
highest, at 3.37. Again, the content tapped by these items was fairly general, although the interface 
for this section was the most difficult (requiring examinees to read information on the screen and 
listen to selected sound files before responding to an item). The item with the lowest p-value in this 
section was one that used speech; a sentence fragment could be played, as could four sentence 
completions. To correctly answer the question, an examinee needed to identify the option that was 
a poor completion. Although the sound files could be played repeatedly, this interface may have 
made the item more difficult than it would have been in written form. Despite its apparent 
difficulty in this instance, it is an appropriate application for tests of dictation or listening in second 
language assessment, and possibly in other applications. 
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Discussion/Summary 

This investigation into a few item innovations found generally positive results. For the 
most part, the technology worked as expected. A continuing development process which uses and 
evaluates the computerized items on the test administration platform should prove to be the best 
guarantee of success in this area. 

Examinees were largely positive about the computer exam. They had few complaints about 
using the test software or about most of the innovations. The incorporation of sound into the test 
proved to be the most radical change, resulting in some very positive examinee comments, along 
with some genuine concerns. While improvements to the audio quality are possible through the use 
of headsets and of better sound files, the more critical issue is appropriate use. Inclusion of sound 
may be beneficial for some content areas (e.g., second language, music) and can provide an aid to 
examinees with certain disabilities (e.g., visual problems, poor reading skills), but should certainly 
not be used in a context where it is not appropriate and/or where it hinders examinees with hearing 
problems. 

The psychometric functioning of the various item types appeared adequate. Examinees 
displayed no difficulty responding to items in Section 2 (where they selected a single best response 
by clicking a box) and in Section 3 (where they indicated a response by clicking on a graphic). A 
few minor problems in Section 4 (where examinees needed to play sound files before responding) 
could be corrected by improvements to the instruction and practice screens. Further investigation 
of the item type used in Section 1 (where examinees were instructed to “Select all that apply”) 
might focus on the effects of guided instruction (e.g., “Select the best 3...”) and of partial-credit 
scoring. 

The development of appropriate forms of assessment is critical, and an essential component 
of the development process is the evaluation of those assessments. While it seems evident that we 
need to expand the types of items and forms of measurement we use, it is equally critical that we 
investigate how well the new forms work. As we creatively develop new procedures, we need to 
ask: what is the important construct or skill that should be measured? will the new assessments 
measure that construct better than traditional measures? will they measure a construct or skill that 




13 



Innovations 13 



cannot be measured through traditional means? and finally, how do the new assessments compare 
to traditional methods in terms of convenience, cost, student preference, and time required to 
conduct and score? As we ask these questions, we will find ways to improve and build on our 
efforts. 
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Appendix A 

Computer-Based Exam Survey 

(with frequency responding to each option) 

1 . What was your level of computer experience before you began this course? 



• very little 48 

• some 44 

® a lot 1 

2. For this class, which type of test would you prefer to take? 

• computer test 60 

• don’t care 16 

• paper-and-pencil test 17 

3. In general, what type of test would you prefer to take? 

• computer test 37 

• don’t care 12 

• paper-and-pencil test 44 

4. Did you have any problems with the Section 1 (educational computing)? 

• no 68 

• yes 25 

5. Did you have any problems with the Section 2 (graphics and images)? 

• no 83 

• yes 10 

6. Did you have any problems with the Section 3 (databases)? 

• no 81 

• yes 12 

7. Did you have any problems with the Section 4 (sound)? 

• no 88 

• yes 5 

8. Did you have any problems with the mouse? 

• no 93 

• yes 0 

9. Did you have any problems with the Previous/Next buttons? 

• no 93 

• yes 0 

10. Do you have any further comments or suggestions? 

• no 70 

• yes 23 



Note: Each “yes” response routed examinees to a screen where they were 
prompted to type in more details about problems or suggestions. 
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Table 1 

Mean Scores on Exam 2 and Course Total by Group 

Computer & Paper-and-Pencil 

Paper-and-Pencil Only 

Examinees Examinees 



Exam 2 

Mean 63.72 60.38 

S.D. 6.56 8.52 

Course Total 

Mean 269.93 256.72 

S.D. 12.43 19.75 



Note: For the Computer and Paper-and-Pencil examinees, N=93; for the 
Paper-and-Pencil Only examinees, due to some missing responses, the N per 
item ranged from 1 73 to 1 76. Exam 2 score is reported as number correct out 
of 80 items. Course Total is reported as number of points out of a possible 
300 (and, for the Computer-examinees, includes 6 points extra credit). 



O 

ERIC 



16 



Innovations 16 



Table 2 

Item P-values for Computer-Based Items, in Computer and Paper-and-Pencil Modes 



Computer Test 



Paper-and-Pencil Test 



Computer & Computer & Paper-and-Pencil 

Paper-and-Pencil Paper-and-Pencil Only 

Examinees Examinees Examinees 

(n=93) (11=93) (n=176) 



11 


.22 


.62 


.63 


12 


.45 






13 


.18 


.72 


.65 


14 


.42 


.84 


.67 


15 


.48 


.86 


.77 


16 


.35 


.62 


.45 


17 


.66 


.77 


.77 


18 


.75 


.92 


.80 


19 


.49 


.86 


.81 


110 


.61 






111 


.97 






112 


.99 






113 


.94 






114 


.62 






115 


.87 






116 


.94 







Note: Some of the computer-based items were adapted from items appearing 
on the paper-and-pencil exam. Where that occurred, p-values for those paper- 
and-pencil items are provided, by group. 




17 



Authorware Professional 

Edit Data Libraries Attributes Text Try It 



visas! 



File 



File 



In this section, items may have more than 1 correct answer : WI 
Please select all correct options-;^ > v- f 



m 



3. Which program(s) listed below are most efficient tor 

developing and presenting slide-show type presentations? ~ f 

• ~ ^ - v^Jg 

■ ' : vE^Hyperefedip W WW-W •. _ . ;.W - W Z '.'1 FT K / i 

" v W>'W. 

IOI " Pnwpr Point ••w ■ • ’’V; ;■ vl -r *£:- • { •' , 



■ 'y>M % 



Power Point W 
I I Linkway Live '• 
O Storyboard: Live; v 
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i* 
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Please select the one best option; 




2. Which of the following is not an 
| advantage of a laser disc? 

□ It is very durable. 

I I It takes very little storage space. 

I □ Its signal can be fed directly into the computer. 

\ j Random access to each individual: frame is available. * vf. 

| □, It-can be stopped. on a single |i^e 1i^|for indefinite period oftime; 



Section 2: Screen 2 of 5 




\°\ 



F* * - • 
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* 0 . 



2 ^is database has been sortediin ascending order 
CHck on the nam&of ithe field used for the soifeFh 



a 


V.. ' -X’. 


-. 7 ; * 


Table: Studentsii.fi it 








V 


Student ID 


1 First Name 


Last Name 


1 Major- 


Class 


Gender 


Aqe 


► 


323-44-1871 


Martina 


Boulet 


Engineering 


Senior 


F 


25 




202-75-9921 


Marion 


Jackson 


Fine Arts 


Senior 


M 


23 




b 7 1 -b'o-3266 


Hannah 


Kirchner 


Education 


Junior 


F 


21 




567-43-51 1 6 


Phii 


Leguin 


Fine Arts 


Junior 


F 


20 


*- 


134-76-8933 


Patrick 


Lenter 


Education 


Junior 


M 


21 


u i 


432-15-5567 


Albert 


Ramirez 


Engineering 


Junior 


M 


19 


J<| 


4 j Record: |1 


| of 6 


\±M~ 
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sounds indig^e|ptt^n select fe by 



1 . 



feedback to a correct response in a computerized lesson 

• •*. -firsts.- • •• 



' -s .... . *_'.v 



! 



□ 


Sound 1 j 




□ 


Sound 2 






□ 


Sound 3 




□ 


Sound 4 j 






□ 


Sound 5 | 



/ 




.■w.'.-.X : 
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